Spectral Batch Normalization: Normalization in the Frequency Domain

Regularization is a set of techniques that are used to improve the generalization ability of deep neural networks. In this paper, we introduce spectral batch normalization (SBN), a novel effective method to improve generalization by normalizing feature maps in the frequency (spectral) domain. The activations of residual networks without batch normalization (BN) tend to explode exponentially in the depth of the network at initialization. This leads to extremely large feature map norms even though the parameters are relatively small. These explosive dynamics can be very detrimental to learning. BN makes weight decay regularization on the scaling factors $\gamma, \beta$ approximately equivalent to an additive penalty on the norm of the feature maps, which prevents extremely large feature map norms to a certain degree. However, we show experimentally that, despite the approximate additive penalty of BN, feature maps in deep neural networks (DNNs) tend to explode at the beginning of the network and that feature maps of DNNs contain large values during the whole training. This phenomenon also occurs in a weakened form in non-residual networks. SBN addresses large feature maps by normalizing them in the frequency domain. In our experiments, we empirically show that SBN prevents exploding feature maps at initialization and large feature map values during the training. Moreover, the normalization of feature maps in the frequency domain leads to more uniform distributed frequency components. This discourages the DNNs to rely on single frequency components of feature maps. These, together with other effects of SBN, have a regularizing effect on the training of residual and non-residual networks. We show experimentally that using SBN in addition to standard regularization methods improves the performance of DNNs by a relevant margin, e.g. ResNet50 on ImageNet by 0.71%.Comment: Accepted by The International Joint Conference on Neural Network (IJCNN) 202

Weight Compander: A Simple Weight Reparameterization for Regularization

Regularization is a set of techniques that are used to improve the generalization ability of deep neural networks. In this paper, we introduce weight compander (WC), a novel effective method to improve generalization by reparameterizing each weight in deep neural networks using a nonlinear function. It is a general, intuitive, cheap and easy to implement method, which can be combined with various other regularization techniques. Large weights in deep neural networks are a sign of a more complex network that is overfitted to the training data. Moreover, regularized networks tend to have a greater range of weights around zero with fewer weights centered at zero. We introduce a weight reparameterization function which is applied to each weight and implicitly reduces overfitting by restricting the magnitude of the weights while forcing them away from zero at the same time. This leads to a more democratic decision-making in the network. Firstly, individual weights cannot have too much influence in the prediction process due to the restriction of their magnitude. Secondly, more weights are used in the prediction process, since they are forced away from zero during the training. This promotes the extraction of more features from the input data and increases the level of weight redundancy, which makes the network less sensitive to statistical differences between training and test data. We extend our method to learn the hyperparameters of the introduced weight reparameterization function. This avoids hyperparameter search and gives the network the opportunity to align the weight reparameterization with the training progress. We show experimentally that using weight compander in addition to standard regularization methods improves the performance of neural networks.Comment: Accepted by The International Joint Conference on Neural Network (IJCNN) 202

High-Resolution Optical Functional Mapping of the Human Somatosensory Cortex

Non-invasive optical imaging of brain function has been promoted in a number of fields in which functional magnetic resonance imaging (fMRI) is limited due to constraints induced by the scanning environment. Beyond physiological and psychological research, bedside monitoring and neurorehabilitation may be relevant clinical applications that are yet little explored. A major obstacle to advocate the tool in clinical research is insufficient spatial resolution. Based on a multi-distance high-density optical imaging setup, we here demonstrate a dramatic increase in sensitivity of the method. We show that optical imaging allows for the differentiation between activations of single finger representations in the primary somatosensory cortex (SI). Methodologically our findings confirm results in a pioneering study by Zeff et al. (2007) and extend them to the homuncular organization of SI. After performing a motor task, eight subjects underwent vibrotactile stimulation of the little finger and the thumb. We used a high-density diffuse-optical sensing array in conjunction with optical tomographic reconstruction. Optical imaging disclosed three discrete activation foci one for motor and two discrete foci for vibrotactile stimulation of the first and fifth finger, respectively. The results were co-registered to the individual anatomical brain anatomy (MRI) which confirmed the localization in the expected cortical gyri in four subjects. This advance in spatial resolution opens new perspectives to apply optical imaging in the research on plasticity notably in patients undergoing neurorehabilitation

External Urethral Sphincter Pressure Measurement: An Accurate Method for the Diagnosis of Detrusor External Sphincter Dyssynergia?

Combined pelvic floor electromyography (EMG) and videocystourethrography (VCUG) during urodynamic investigation are the most acceptable and widely agreed methods for diagnosing detrusor external sphincter dyssynergia (DESD). Theoretically, external urethral sphincter pressure (EUSP) measurement would provide enough information for the diagnosis of DESD and could simplify the urodynamic investigation replacing combined pelvic floor EMG and VCUG. Thus, we evaluated the diagnostic accuracy of EUSP measurement for DESD. PATIENTS #ENTITYSTARTX00026;A consecutive series of 72 patients (36 women, 36 men) with neurogenic lower urinary tract dysfunction able to void spontaneously was prospectively evaluated at a single university spinal cord injury center. Diagnosis of DESD using EUSP measurement (index test) versus combined pelvic floor EMG and VCUG (reference standard) was assessed according to the recommendations of the Standards for Reporting of Diagnostic Accuracy Initiative.Using EUSP measurement (index test) and combined pelvic floor EMG and VCUR (reference standard), DESD was diagnosed in 10 (14%) and in 41 (57%) patients, respectively. More than half of the patients presented discordant diagnosis between the index test and the reference standard. Among 41 patients with DESD diagnosed by combined pelvic floor EMG and VCUR, EUSP measurement identified only 6 patients. EUSP measurement had a sensitivity of 15% (95% CI 5%-25%), specificity of 87% (95% CI 76%-98%), positive predictive value of 60% (95% CI 30%-90%), and negative predictive value of 56% (95% CI 44%-68%) for the diagnosis of DESD.For diagnosis of DESD, EUSP measurement is inaccurate and cannot replace combined pelvic floor EMG and VCUR

Infliximab biosimilar-to-biosimilar switching in patients with inflammatory rheumatic disease:clinical outcomes in real-world patients from the DANBIO registry

OBJECTIVE: Successful uptake of biosimilars in rheumatology is limited by lack of real-world evidence regarding effectiveness of biosimilar-to-biosimilar switching. We investigated infliximab biosimilars CT-P13-to-GP1111 switching among patients with rheumatoid arthritis (RA), psoriatic arthritis (PsA) and axial spondyloarthritis (AxSpA). METHODS: Observational cohort study from the DANBIO registry. Patients were classified as originator-naïve or originator-experienced. Retention rates of 1-year GP1111 treatment were explored (Kaplan-Meier). We identified baseline factors (at the time of switch) associated with withdrawal of GP1111 (multivariable Cox-regression analyses with HRs including originator treatment history). Changes in subjective and objective measures of disease activity 4 months before and after the switch were assessed in individual patients. RESULTS: Of 1605 patients (685 RA, 314 PsA and 606 AxSpA, median disease duration was 9 years, 37% in Clinical Disease Activity Index/Ankylosing Spondylitis Disease Activity Score remission), 1171 were originator-naïve. Retention rates at 1-year were 83% (95% CI: 81% to 85%) and 92% (95% CI: 90% to 95%) for the originator-naïve and originator-experienced, respectively. GP1111 retention rates were higher in originator-experienced compared to originator-naïve with RA (HR=0.4 (95% CI: 0.2 to 0.7)) and PsA (HR=0.2 (95% CI: 0.1 to 0.8)), but not significantly for AxSpA: HR=0.6 (95% CI: 0.3 to 1.2). Lower disease activity was associated with higher retention. Changes in disease activity preswitch and postswitch were close to zero. CONCLUSION: This real-world observational study of more than 1600 patients with inflammatory arthritis showed high 1-year retention following a nationwide infliximab biosimilar-to-biosimilar switch. Retention was higher in originator-experienced and in patients with low disease activity, suggesting outcomes to be affected by patient-related rather than drug-related factors

31st Annual Meeting and Associated Programs of the Society for Immunotherapy of Cancer (SITC 2016) : part two

Background The immunological escape of tumors represents one of the main ob- stacles to the treatment of malignancies. The blockade of PD-1 or CTLA-4 receptors represented a milestone in the history of immunotherapy. However, immune checkpoint inhibitors seem to be effective in specific cohorts of patients. It has been proposed that their efficacy relies on the presence of an immunological response. Thus, we hypothesized that disruption of the PD-L1/PD-1 axis would synergize with our oncolytic vaccine platform PeptiCRAd. Methods We used murine B16OVA in vivo tumor models and flow cytometry analysis to investigate the immunological background. Results First, we found that high-burden B16OVA tumors were refractory to combination immunotherapy. However, with a more aggressive schedule, tumors with a lower burden were more susceptible to the combination of PeptiCRAd and PD-L1 blockade. The therapy signifi- cantly increased the median survival of mice (Fig. 7). Interestingly, the reduced growth of contralaterally injected B16F10 cells sug- gested the presence of a long lasting immunological memory also against non-targeted antigens. Concerning the functional state of tumor infiltrating lymphocytes (TILs), we found that all the immune therapies would enhance the percentage of activated (PD-1pos TIM- 3neg) T lymphocytes and reduce the amount of exhausted (PD-1pos TIM-3pos) cells compared to placebo. As expected, we found that PeptiCRAd monotherapy could increase the number of antigen spe- cific CD8+ T cells compared to other treatments. However, only the combination with PD-L1 blockade could significantly increase the ra- tio between activated and exhausted pentamer positive cells (p= 0.0058), suggesting that by disrupting the PD-1/PD-L1 axis we could decrease the amount of dysfunctional antigen specific T cells. We ob- served that the anatomical location deeply influenced the state of CD4+ and CD8+ T lymphocytes. In fact, TIM-3 expression was in- creased by 2 fold on TILs compared to splenic and lymphoid T cells. In the CD8+ compartment, the expression of PD-1 on the surface seemed to be restricted to the tumor micro-environment, while CD4 + T cells had a high expression of PD-1 also in lymphoid organs. Interestingly, we found that the levels of PD-1 were significantly higher on CD8+ T cells than on CD4+ T cells into the tumor micro- environment (p < 0.0001). Conclusions In conclusion, we demonstrated that the efficacy of immune check- point inhibitors might be strongly enhanced by their combination with cancer vaccines. PeptiCRAd was able to increase the number of antigen-specific T cells and PD-L1 blockade prevented their exhaus- tion, resulting in long-lasting immunological memory and increased median survival

Gestaltung und Integration von Arbeitsplanungskompetenzen für hierarchielose Produktionsnetze

Die vorliegende Arbeit leistet einen Beitrag zur Überführung der Vision hierarchieloser Produktionsnetze in die betriebliche Praxis. Dazu wurden speziell für den Problembereich der Arbeitsplanung Untersuchungen durchgeführt, damit auf der Grundlage von speziell erarbeiteten Konzepten, Methoden und Systematiken ein effizientes Funktionieren dieser innovativen Organisationsform für kooperierende KMU ermöglicht wird. Im Resultat er-weisen sich hierarchielose Produktionsnetze aus Sicht der Arbeitsplanung als eine geeigne-te Vernetzungsform, um die in der Ausgangssituation beschrieben existierenden Defizite in der betrachteten Unternehmenskategorie zu minimieren